Skip to content

Conversation

@pjanotti
Copy link
Contributor

@pjanotti pjanotti commented Oct 13, 2025

Fix the generate target so developers using Windows can do make generate for their components on Windows. Main things in this change:

  1. Use of path that can be pre-pended to PATH, the one being used contains : and even escaping it is not enough to get it to correctly work when it is being included in PATH
  2. Ensure that on Windows all tools used by go generate have the exe extension. This is needed because the cmd package used by go generate checks for the known executable extensions on Windows. Notice that make and Git Bash don't require that and same extension is not needed for the other tools.
  3. The generate target now triggers building the tools it needs as various other targets so it can avoid the full install-tools.

Future work:

  • go.* targets are really slow on Windows since they trigger the re-evaluation of all shell commands that are really expensive on Windows. Basically any recursive make call from for-all is currently too expensive on Windows.
  • Move the definition of components custom tools to the respective components, possibly using the new go.mod tool directive. For now I opted to keep the tool used by githubreceiver as it is.

@pjanotti pjanotti requested a review from a team as a code owner October 13, 2025 17:28
@pjanotti pjanotti added the ci-cd CI, CD, testing, build issues label Oct 13, 2025
@pjanotti pjanotti requested a review from fatsheep9146 October 13, 2025 17:28
@pjanotti pjanotti added the Run Windows Enable running windows test on a PR label Oct 13, 2025
.PHONY: for-all
for-all:
@set -e; for dir in $(ALL_MODS); do \
@set -e; for dir in $${ALL_MODS}; do \
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is needed given the limits for command-line length on Windows. This way ALL_MODS is passed as an environment variable instead of being expanded in place.

@atoulme atoulme merged commit a627ba7 into open-telemetry:main Oct 14, 2025
239 checks passed
@github-actions github-actions bot added this to the next release milestone Oct 14, 2025
@pjanotti pjanotti deleted the fix-make-generate-on-windows branch October 14, 2025 19:29
@jade-guiton-dd
Copy link
Contributor

jade-guiton-dd commented Oct 15, 2025

It looks like this PR broke the make for-all target, which in turn broke the contrib-tests over on the core repo.

I believe the problem is that $(ALL_MODS) was switched to $${ALL_MODS}, which references the ALL_MODS environment variable, which is never defined. (ALL_MODS is a Make variable, which are not automatically injected as environment variables in commands by default.) The result is that the for loop does nothing. This can be confirmed by running make for-all CMD='echo test'.

@jade-guiton-dd
Copy link
Contributor

Filed #43572 which should hopefully fix this. The ALL_MODS variable seems to fit within the less restrictive 32768 byte limit for environment variables on Windows, but could you confirm that things still work on Windows after the fix @pjanotti?

songy23 pushed a commit that referenced this pull request Oct 15, 2025
#### Description

#43509 changed the `make for-all` Makefile target to access the
`ALL_MODS` variable through environment variables instead of expanding
it into the shell command, in order to bypass Windows' 8192 byte command
line length limit.

However, the environment variable was not defined, which led to `make
for-all` becoming a no-op. This can be confirmed by running `make
for-all CMD='echo test'`.

(I discovered this issue because [this
PR](open-telemetry/opentelemetry-collector#13996)
in core repository, which should be failing `contrib-tests`, suddenly
started passing them. This was because the job inserts `replace`
statements in a copy of contrib using `for-all`; failing to do that
caused the contrib tests to run against whatever version of core is
imported here rather than against the PR's code.)

This PR fixes that oversight, by adding the `export` keyword to the
`ALL_MODS` variable.
@pjanotti
Copy link
Contributor Author

pjanotti commented Oct 15, 2025

ops, sorry about that @jade-guiton-dd and thanks for the fix. My validation of this was bad, kind of completed an "empty" generate 🤦🏼

The generate from repo root is broken on Windows, but not due to fix #43572. Anyway, I can live with that, the main thing with my change was to be able to generate from the component. I will look later into fixing make generate from the root in Windows.

ChrsMark pushed a commit to ChrsMark/opentelemetry-collector-contrib that referenced this pull request Oct 20, 2025
Fix the `generate` target so developers using Windows can do `make
generate` for their components on Windows. Main things in this change:

1. Use of path that can be pre-pended to `PATH`, the one being used
contains `:` and even escaping it is not enough to get it to correctly
work when it is being included in `PATH`
2. Ensure that on Windows all tools used by `go generate` have the `exe`
extension. This is needed because the `cmd` package used by `go
generate` checks for the known executable extensions on Windows. Notice
that `make` and Git Bash don't require that and same extension is not
needed for the other tools.
3. The `generate` target now triggers building the tools it needs as
various other targets so it can avoid the full `install-tools`.

Future work:
- `go.*` targets are really slow on Windows since they trigger the
re-evaluation of all shell commands that are really expensive on
Windows. Basically any recursive make call from `for-all` is currently
too expensive on Windows.
- Move the definition of components custom tools to the respective
components, possibly using the new go.mod `tool` directive. For now I
opted to keep the tool used by `githubreceiver` as it is.
ChrsMark pushed a commit to ChrsMark/opentelemetry-collector-contrib that referenced this pull request Oct 20, 2025
#### Description

open-telemetry#43509 changed the `make for-all` Makefile target to access the
`ALL_MODS` variable through environment variables instead of expanding
it into the shell command, in order to bypass Windows' 8192 byte command
line length limit.

However, the environment variable was not defined, which led to `make
for-all` becoming a no-op. This can be confirmed by running `make
for-all CMD='echo test'`.

(I discovered this issue because [this
PR](open-telemetry/opentelemetry-collector#13996)
in core repository, which should be failing `contrib-tests`, suddenly
started passing them. This was because the job inserts `replace`
statements in a copy of contrib using `for-all`; failing to do that
caused the contrib tests to run against whatever version of core is
imported here rather than against the PR's code.)

This PR fixes that oversight, by adding the `export` keyword to the
`ALL_MODS` variable.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-cd CI, CD, testing, build issues Run Windows Enable running windows test on a PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants